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Abstract: 
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An item model serves as an explicit representation of the variables in an assessment 
task. An item model includes the stem, options, and auxiliary information. The stem is the 
part of an item which formulates context, content, and/ or the question the examinee is 
required to answer. The options contain the alternative answers with one correct option 
and one or more incorrect options or distractors. The auxiliary information includes any 
additional material, in either the stem or option, required to generate an item, including 
texts, images, tables, and/or diagrams. In this study, we first present a taxonomy for item 
model development where variables in the stem are crossed with variables in the options 
to create a matrix of possible item model types. We then provide examples of each stem- 
by-option combination. Finally, we develop a software engine and apply the software to 
each item model type to generate multiple instances for each model. 
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Introduction 

Developments in cognitive science, mathematical statistics, computer 
technology, educational psychology, and computing science are creating 
opportunities for theoretical and practical changes in educational mea- 
surement, and related assessment fields. One consequence of these inter- 
disciplinary influences is the emergence of a new area of research called 
assessment engineering (Luecht, 2006a, 2006b, 2007). Assessment engi- 
neering is an innovative approach to measurement where engineering- 
based principles are used to direct the design and development as well 
as the analysis, scoring, and reporting of assessment results. With this 
approach, the measurement specialist begins by defining the construct 
of interest using specific, empirically-derived cognitive models of task 
performance. Next, item models are created to produce replicable assess- 
ment tasks. Finally, psychometric models are applied to the examinee 
response data collected using the item models to produce scores that are 
both replicable and interpre table. 

Assessment engineering differs from more traditional approaches to 
test development and analysis in four fundamental ways. First, cognitive 
models guide item development, rather than content blueprints. Hence, 
the assessment principles used in test construction are much more specific 
allowing items to be created quickly and efficiently during the development 
cycle. Second, explicit item models are created to control and manipulate 
both the content and difficulty of the items. Content experts use the item 
models during development thereby producing assessment tasks that 
adhere to strict quality controls and that meet high psychometric stan- 
dards. Third, automated test assembly procedures are employed to build 
assessments that function to exacting specifications. As a result, multiple 
test forms can be created from a bank of items very efficiently according to 
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both content and statistical specifications. Fourth, psychometric models 
are employed in a confirmatory - versus exploratory - manner to assess 
the model-data fit relative to the intended underlying structure of the con- 
structs or traits the test is designed to measure. The outcomes from these 
model-data fit analyses also provide developers with guidelines for specific 
modifications to the cognitive and item models, as needed, to facilitate 
the acquisition of data that supports the intended assessment inferences 
(Luecht, Gierl, Tan, & Huff, 2006). 

The purpose of this study is to describe and illustrate an approach for 
developing item models, which is the second stage in the assessment engi- 
neering framework. We introduce a taxonomy of item model types intended 
to help developers identify new models and methods for producing high- 
quality assessment items. We also illustrate how technology can be used 
with these models to generate large numbers of test items. 

Introduction to Item Modeling: 

Terminology and Concepts 

An item model^ (LaDuca, Staples, Templeton, & Holzman, 1986; Bejar, 
1996, 2002; Bejar, Lawless, Morley, Wagner, & Bennett, 2003) serves as 
an explicit representation of the variables in an assessment task, which 
includes the stem, the options, and oftentimes auxiliary information. The 
stem is the part of an item which formulates context, content, and/or the 
question the examinee is required to answer. The options contain the alter- 
native answers with one correct option and one or more incorrect options 
or distractors. When dealing with a multiple-choice item model, both stem 
and options are required. With an open-ended or constructed-response 
item model, only the stem is created. Auxiliary information includes any 
additional material, in either the stem or option, required to generate an 
item, including texts, images, tables, and/or diagrams. 

The stem and options can be divided further into elements. These 
elements are often denoted as strings, S, which are non-numeric values 
and integers, I, which are numeric values. This terminology is adopted 
from Bejar et al. (2003). By systematically manipulating these elements, 
measurement specialists can generate large numbers of instances or items 
for each model. If the instances are intended to measure content at similar 
difficulty levels, then the generated items are isomorphic^. When the goal 
of item generation is to create isomorphic instances, the measurement 
specialist manipulates the incidental elements, which are the surface fea- 
tures of an item that do not alter item difficulty. Conversely, if the instances 
are intended to measure content at different difficulty levels, then the 
generated items are variants. When the goal of item generation is to 
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create variant instances, the measurement specialist can manipulate the 
incidental elements, but must manipulate one or more radical elements 
in the item model. The radicals are the deep features that alter item 
difficulty, and may also affect the psychometric properties of the test such 
as dimensionality. 

To illustrate these concepts, two examples from Grade 6 mathematics 
are presented. In both examples, the item model is represented as the stem 
and options variables. The stem of the first example contains two integers 
(II, 12) while the stem of the second example contains two additional 
strings (SI, S2). The II element includes Anns payment. It ranges from 
$1525 to $1675 in increments of $75. The 12 element includes the cost of 
the lawn, as either $30/m^ or $45/m^. As the first example represents only 
one lawn shape, shape is fixed and, therefore, no string variable is required 
(Figure la, next page). The four alternatives, labelled A to D, are gener- 
ated in the example using algorithms produced from the integer values 
II and 12 as well as from keyed option A. In the second example shown in 
Figure lb (page 8), the shape variable includes a square and a circle in the 
item stem. As a result, the SI element describes the shape of the lawn and 
S2 must be constrained to match SI. Hence, the S2 element presents the 
appropriate area concept, side length or radius, required for calculating 
the area of the shape. Because the area calculation for a square differs from 
a circle, the options are also expected to include algorithms for computing 
the area of a circle (right-side of options box) in addition to the area of a 
square (left-side of options box). The area calculation difference between 
a square and a circle serves as the radical for the Figure lb example. There 
is no auxiliary information for this item model (see Bejar et al., 2003, p. 9, 
for another example in mathematics). 


Developing a Taxonomy of Item Model Types to Promote Assessment Engineering 


Gierl, Zhou, Alves 


Figure la: 


Stem 


Elements 


Options 


Key 


Item Model in Mathematics Used to Generate Isomorphic Instances 

Ann has paid $ 1 525 for planting her lawn. The cost of lawn is $45/m^. Given 
the shape of her lawn is square, what is the side length of Ann's lawn? 

A. 5.8 

B. 6.8 

C. 4.8 

D. 7.3 


Item Model Variables 

Ann has paid $[n]for planting her lawn. The cost of lawn is $ 12/m^. Given 
the shape of her lawn is square, what is the side length of Ann's lawn? 


[H] Value Range: 1525-1 675 by 75 
[i^ Value Range: 30 or 40 
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Figure 1 b: Item Model in Mathematics Used to Generate Variant Instances 


Item Model Variables 


Stem 


Elements 


Options 


Key 


Ann has paid $|T^for pla 
the shape of her lawn is 

mti 

SI 

ng her lawn. The cost of lawn is $ 12/m^. Given 
, what is the[^ of Ann's lawn? 



[n] Value Range: 1525 -1 675 by 75 
[1T| Value Range: 30 or 40 


[sT| Range:"square"or"circular" 

Range: "side length" or "radius" 

As SI = "square", then S2 = "side length" 
As SI = "circular" then S2 = "radius" 


AsSI = "square" 

AsSI = "circular" 

A. = a/^V/2 

A. = 

B. = a/^V/2 + 1 

B.= 

C= a/^V/2 

C= VV/2'3-14 -1 

D.= VTT + 1'5 

D.= V'/;2*3-14 +1.5 


A 


J T L-A r 
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Some Benefits of Item Modeling 

Traditional item development using manual processes can be ineffi- 
cient, largely because items are treated as isolated entities that are individ- 
ually created, reviewed, and formatted. Because the items are individually 
authored, they yield unpredictable statistical outcomes (and, therefore, 
require field testing) because the incidental and radical elements are not 
easily identified or well understood. Traditional item development can 
also pose security risks for a testing program because the costs associated 
with construction, calibration, and maintenance limit the number of oper- 
ational items that are available at any one time — with fewer operational 
items available, exposure risks may increase because more examinees are 
being exposed to each item. Drasgow, Luecht, and Bennett (2006, p. 473), 
in their seminal chapter in Educational Measurement (4^^ Edition) on tech- 
nology and testing, provide this summary: 

The demand for large numbers of items is challenging to satisfy 
because the traditional approach to test development uses the item 
as the fundamental unit of currency. That is, each item is individually 
hand-crafted - written, reviewed, revised, edited, entered into a 
computer, and calibrated - as if no other like it had ever been created 
before. A second issue with traditional approaches is that it is 
notoriously hard to hit difficulty targets, which results in having too 
many items at some levels and not enough at other levels. Finally, 
the pretesting needed for calibration in adaptive testing programs 
entails significant cost and effort. 

Item modeling can help overcome some of the limitations of the tra- 
ditional approach thereby enhancing test development in two important 
ways. First, item modeling is cost-effective. The purpose of development 
is to create multiple models, where each model yields many items. Hence, 
banks can be created quickly which will minimize item exposure because 
larger pools of operational items are available for each test administra- 
tion. The logic behind item modeling can also lead to more cost-effective 
practices because items are treated as classes which require a systematic 
and strategic development approach compared with treating each item 
as a single unit. Hence, the cost per item is lower because the unit of 
analysis is multiple instances per model rather than single instances per 
content specialist. Also, costly, yet common, errors in item development - 
including omissions or additions of words, phrases, or expressions as well 
as spelling, punctuation, capitalization, item structure, typeface, format- 
ting, and language (e.g., English to French translation) problems - can be 
avoided because only specific elements in the stem and options are manip- 
ulated across large numbers of items. That is, the item model serves as a 
template where content specialists manipulate specific, well-defined, ele- 
ments. The remaining components in the template, once finalized, are not 
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altered during item development. As a result, item modeling should allow 
content specialists to quickly create large numbers of high-quality opera- 
tional items that require few revisions during the development stage. 

Second, item models provide the foundation necessary for automatic 
item generation. Automatic item generation is a procedure for using item 
models to create isomorphic instances with known item characteristics, 
often in real-time, as the examinee is writing the test. The procedure has 
two requirements: An item class must be described in enough detail to 
permit a computer to create instances of the class automatically. Also, the 
variables that affect item difficulty must be controlled across instances so 
the generated items do not require separate calibration (Drasgow et al., 
2006). One key benefit of automatic item generation is that it minimizes, 
if not eliminates, the need for extensive field testing because the isomor- 
phic instances generated from the parent model are pre-calibrated and, 
thus, do not need to be field tested. Automatic item generation can pro- 
ceed from either strong or weak theory. If strong theory is used, calibrated 
items are generated automatically using the design principles articulated 
in a cognitive model (i.e., step 1 in assessment engineering framework; 
see also Leighton & Gierl, 2007). The cognitive model provides a detailed 
description of the variables that affect examinee performance which, in 
turn, can help pinpoint the item difficulty features. The obvious benefit 
of strong theory is that the cognitive features of item performance are 
identified and articulated in such detail that difficulty can be predicted and 
controlled. Unfortunately, few strong theories currently exist to guide our 
educational and psychological measurements. As a result, strong theory 
for automatic item generation has been limited to specific tasks in domains 
such as mental rotation (Bejar, 1990) and spatial ability (Embretson & 
Yang, 2007). 

In the absence of strong theory, weak theory must be used. Weak 
theory yields calibrated items generated automatically using design 
guidelines (rather than design principles) discerned from a combina- 
tion of experience, theory, and research (rather than cognitive models) 
(Drasgow et al., 2006). Initially, the guidelines are used to identify a parent 
item model. Then, incidental item features in the parent item model are 
manipulated to produce isomorphic instances. The benefit of weak theory 
for automatic item generation stems from its practicality. Parent models 
can often be identified by reviewing items from previously administered 
exams. Weak theory is also well-suited to broad content domains where 
few theoretical descriptions exist about the cognitive knowledge and skills 
used by examinees to solve items. The main drawback of weak theory is 
that item difficulty is neither predictable nor easily controlled. However, if 
data for a parent item model are available, statistical procedures have been 
developed to account for the variation among the isomorphic instances 
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and estimate their item difficulty levels (e.g., Glas & van der Linden, 2003; 
Mislevy, Wingersky, & Sheehan, 1994; Sinharay, Johnson, & Williamson, 
2003; Sinharay & Johnson, 2005) . In short, item modeling can enhance test 
development practices and provide the necessary foundation for sophisti- 
cated psychometric procedures such as automatic item generation. 

To create item models systematically and strategically using either 
strong or weak theory, a generic item model taxonomy is required. This 
type of taxonomy, in fact, is a prerequisite for a functional automatic item 
generation system because it provides the guiding principles necessary for 
designing a large number of diverse item models by outlining their struc- 
ture, function, similarities, differences, and limitations. Unfortunately, 
the educational and psychological measurement literature contains little 
discussion on how to develop item models and few examples exist. 
Therefore, our first step is to create a taxonomy of item model types that 
could generalize across content areas, as a way of offering test develop- 
ment principles for creating item models. We also provide examples of 
these item model types. 

A Taxonomy for Designing Item Models 

A taxonomy for item model development requires at least three vari- 
ables: the stem, options, and auxiliary information. Each variable func- 
tions differently. The stem is the section of the model used to formulate 
context, content, and/or questions. It contains four categories, as shown 
in Figure 2. Independent indicates that the n, element(s) (n, > 1) in the 
stem are independent or unrelated to one another. That is, a change in one 
element will have no affect on the other stem elements in the item model. 
Dependent indicates n^ element(s) > 2) in the stem are dependent or 
directly related to one other. Mixed Independent/Dependent include both 
independent (n, > 1) and dependent (n^ > 1) elements in the stem. Fixed 
represents a constant stem format with no variation or change. 

Figure 2: Categories in the Item Model Stem 



Independent element(s) 


Dependent element(s) 


J 


Mixed Independent/Dependent element(s) ] 


Fixed element(s) 
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The optionSy shown in Figure 3, contain the alternatives for the item 
model when the multiple-choice format is used. The options contain three 
categories. Randomly-selected options refers to the manner in which the 
distractors are selected from their corresponding content pools. The dis- 
tractors are selected randomly. Constrained options mean that the keyed 
option and the distractors are generated according to specific constraints, 
such as formulas, calculation, and/or context. Fixed options occur when 
both the keyed option and distractors are invariant or unchanged in the 
item model. 

Figure 3: Categories in the Item Model Options 



By crossing the stem and options categories, a matrix of item model 
types can be produced. The stem-by-options matrix is presented in Table 1. 
Ten functional combinations are designated with a checkmark, ‘V”. The 
two remaining combinations are labelled not applicable, “NA”, because a 
model with a fixed stem and constrained options is an infeasible item type 
and a model with a fixed stem and options produces a single multiple- 
choice item type (i.e., a traditional multiple-choice item). 

Table 1 : Plausible Stem-by-Option Combinations in the Item Model 

Taxonomy 


Options 

stem 

Independent 

Dependent 

Mixed 

Fixed 

Randomly Selected 





Constrained 

Y 



N/A 

Fixed 

Y 



N/A 


Next, the ten stem-by-options combinations are illustrated. We draw on 
examples, first, from mathematics to demonstrate the applicability of our 
taxonomy. For each stem-by-option combination in Table 1, we present 
an item, followed by the item model template which outlines the stem, 
elements, options, auxiliary information, and key (cf. Bejar et al., 2003, 
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Model #1: 


Stem 

Elements 

Options 

Auxiliary 

Information 

Key 


p. 9). These 10 models will then be used with the item generator software 
described in the next section to create new items. We also provide a second 
set of examples in the Appendix. These examples were drawn from diverse 
content areas, including science, social studies, language arts, and archi- 
tecture. 

Stem: Independent; Options: Randomly Selected; 

Auxiliary Information: None 

The students could see a circular lake from the top of a Tramway. 

The distance around the lake is known as its 

A. circumference 

B. diameter 

C. radius 

D. area 


Item Model Variables 

Isil could see a circular S2 from the top of a 0. 
The distance around the S2 is known as its 


[si Range: "Some students", "Bob and Mike", "Anne and her sister", 
"Some boys", "Some girls" 

S2 Range: "lake", "pool" 

Range: "Tramway", "mountain", "building", "tower" 


Key : circumference or perimeter 

Distractors : diameter, radius, area, sector, chord, arc 


None 


A 
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Model #2: 


Stem 


Elements 

Options 

Auxiliary 

Information 

Key 


Stem: Independent; Options: Constrained; 
Auxiliary Information: Diagram 


This is a diagram of the Pizza Place's 
floor plan. Fire regulations state that 
each customer in a dining room 
must have a minimum of 2.2 m^ of 
floor space. What is the maximum 
number of customers that can be 
seated in the Pizza Place's dining 
room when x= 3.0, y= 5.0, 4.0, 

and the restaurant is 1 0 x 1 6? 

A. 47 

B. 59 

C. 54 

D. 72 



-X - 


y 

Patio 





Dining Room 

Kitchen 




— z — 


Item Model Variables 


This is a diagram of the Pizza Place's 
floor plan. Fire regulations state that 
each customer in a dining room 
must have a minimum of 2.2 m^ of 
floor space. What is the maximum 
number of customers that can be 
seated in the Pizza Place's dining 
room when x ^Tl,y=az=[ill+ 1, 
and the restaurant is 2^1^ by 

[n]+ 2 *[12]+ 3 ? 

[iT| Value Range: 1 -3 by 1 
[1^ Value Range: 1 0-1 8 by 1 



-X — 


y 

Patio 





Dining Room 

Kitchen 




— z — 


A. Round Down (4 4 *^-^*^)/ 2.2 

B. Round Down (2 (□+ 2 *\^+ 3) -\^*\^) / 2.2 

C. Round Down (4 •|i2|-|j2]+4-|i2l)/2.2 

D. Round Down (2*[iJ*[i^+4*[]^*[l^+6^)/2.2 


Diagram of a Pizza Place's floor plan 


A 
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Model #3: 


Stem 


Elements 


Options 

Auxiliary 

Information 

Key 


Stem: Independent; Options: Fixed; Auxiliary Information: None 

In order to make a particular shade of green paint, Mary uses 24 parts of 
blue pigment, 12 parts of white, and 6 parts of yellow. What is the simplest 
ratio of these pigments? 

A. 4:2:1 

B. 6:3:2 
C 12:6:3 

D. 1:1/2:174 


Item Model Variables 

In order to make a particular shade of^ paint, Mary uses 4*[TT| parts of S2 
pigment, 2*[n] parts of white, andfll] parts of^. What is the simplest ratio 
of these pigments? 


m Range: "green", "orange", "purple", "brown" 
S2 Range: "blue", "red" 

^ Range: "yellow", "blue", "black" 

[iT] Value Range: 2, 3, 6, or 12 

As SI ="green", S2="blue" S3="yellow"; 

As Sl="orange", S2="red", S3="yellow"; 

As SI ^"purple", S2="red" S3="blue"; 

As Sl="brown", S2="red", S3="black" 



None 


A 


J T L-A r 
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Model #4: 


Stem 


Elements 


Options 


Auxiliary 

Information 


Key 


Stem: Dependent; Options: Randomly Selected; 
Auxiliary Information: Pictures 

Some students visited the sights 
around Jasper. Omar observed that 
the JasperTramway formed an angle 
with the surroundings. 

What kind of angle is Z.ABD? 

A. Acute 

B. Right 

C. Obtuse 

D. Straight 



Item Model Variables 


Some students visited |sj. Omar 
observed that the S2 formed an 
angle with the surroundings. 

What kind of angle is Z.ABD? 


A 



B D 


|s| Range: "the sights around Jasper", "a pizzeria", "a park", "a watch store" 
S2 Range: "JasperTramway", "pizza slice", "park's table", "a clock" 

As Sl = "the sights around Jasper", S2= "Jasper Tramway" 

As Sl = "a pizzeria", then S2= "pizza slice" 

AS S1 = "a park", then S2= "park's table" 

AS SI ="a watch store", then S2="clock's pointer" 


Key : Acute 

Distractors : Right, Obtuse, Straight, Vertical 



Picture of JasperTramway; Picture of pizza slice; Picture of park's table; 
Picture of a clock 
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Model #5: 


Stem 


Elements 


Options 


Auxiliary 

Information 


Stem: Dependent; Options: Constrained; 
Auxiliary Information: Picture 


The thermostat of an oven malfunctioned. 
First, the temperature dropped 5°C, then it 
increased 7°C, fell 1 2°C, and finally decreased 
a further 30°C before it stabilized 185°C. 
What was the original temperature? 

A. 239°C 

B. 225°C 

C. 131°C 

D. 145°C 



Item Model Variables 


The thermostat of an oven malfunctioned. 
First, the temperature dropped[n]°|^, then 
it increased [1^° Hi], fell[l^^, and finally 
decreased a further[]^^1 before it stabilized 
[|^°[^. What was the original temperature? 



As^="°C" 

[TT] Value range: 3 to 1 8 by 3 
[1^ Value range: 2 to 20 by 2 
jl3| Value range: 5 to 1 5 by 1 
14 Value range: 1 0 to 40 by 4 
[T^ Value range: 1 00 to 200 by 5 


AsS1 = "°F" 

[TT| Value range: 1 5 to 30 by 3 
[i^ Value range: 1 0 to 30 by 2 
[l3| Value range: 21 to 30 by 1 
[1^ Value range: 50 to 60 by 5 
[T^ Value range: 200 to 300 by 5 


A. [n]+^+l+0+[i5] 

B. [n]-§+[i^+0+[i^ 

c. [n]+[il-i]-0+[il] 

D. 


Oven picture 


B 


Key 
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Model #6: 


Stem 


Elements 


Options 

Auxiliary 

Information 

Key 


Stem: Dependent; Options: Fixed; Auxiliary Information: None 

To calculate the speed of a motorcycle, ifd = distance a motorcycle travels 
in metres, r= speed in m/s, and f = time in seconds, the formula d = rt 
would have to be rewritten as 

A. r-d-t 

B. r-dt 
C r=c//f 
D. r=t/d 


Item Model Variables 

To calculate the speed of a[Sl], if d = distance a[s] travels in S2, r= speed in 
i? / S4, and f = time in the formula d = rf would have to be rewritten as 


1^ Range: "motorcycle", "bike", "car", "truck" 

S2 Range: "metres", "kilometres" 

Sj Range: "m", "km" 

S4 Range: "s","h" 
sj Range: "seconds", "hours" 

As SI = "bike", then S2= "metres" 

As SI = "motorcycle", "car", "truck", then S2="kilometres" 
As S2= "metres", then S3="m", S4="s", S5="seconds" 

As S2= "kilometres", then S3="km", S4="h" S5="hours" 



None 


C 


J T L-A r 
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Model #7: 


Stem 


Elements 


Stem: Mixed; Options: Randomly Selected; 

Auxiliary Information: None 

Four of the students had a foot race at their campsite near Jasper. John 
finished 5 s behind Ryan, Sheila finished 3 s behind John, Danielle was 
6 s in front of Sheila. 

In what order, from first to last, did the students finish? 

A. Ryan, Danielle, Sheila, John 

B. Ryan, John, Danielle, Sheila 

C. Ryan, Sheila, John, Danielle 

D. Ryan, Danielle, John, Sheila 


Item Model Variables 

Four^ had a S2 at their^. John finished [m] S4 behind Ryan, Sheila 
finished [i^S4 behind John, Danielle was[l^ S4 in front of Sheila. 

In what order, from first to last, did the[Sl| finish? 
m Range: "students", "kids", "children" 

S| Range: "foot race", "bike race", "competition", "raffle basket competition", 
"Miniature GolfTournament" "balloon race", "Candy Bar Bingo" 

^ Range: "school", "campsite near Jasper", "community league" 

S4 Range: "s", "points"" 

[n] 3 to 6 by 1 
g 2 to 5 by 1 

g 12+2 

As S2="foot raceV'bike race", or"balloon race", then S4="s" 

As S3="raffle basket competition", "Miniature GolfTournament", or"Candy 
Bar Bingo", then S4="points" 
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Options 


Key : Ryan, Danielle, John, Sheila 
Distractors : 

Danielle, Ryan, Sheila, John 
Danielle, John, Ryan, Sheila 
Danielle, John, Sheila, Ryan 
Danielle, Sheila, Ryan, John 
Danielle, Sheila, John, Ryan 
Danielle, Ryan, John, Sheila 
Ryan, Danielle, Sheila, John 
Ryan, John, Danielle, Sheila 
Ryan, John, Sheila, Danielle 
Ryan, Sheila, Danielle, John 
Ryan, Sheila, John, Danielle 
John, Ryan, Sheila, Danielle 
John, Ryan, Danielle, Sheila 
John, Danielle, Ryan, Sheila 
John, Danielle, Sheila, Ryan 
John, Sheila, Danielle, Ryan 
John, Sheila, Ryan, Danielle 
Sheila, Danielle, Ryan, John 
Sheila, Danielle, John, Ryan 
Sheila, Ryan, Danielle, John 
Sheila, Ryan, John, Danielle 
Sheila, John, Danielle, Ryan 
Sheila, John, Ryan, Danielle 


Auxiliary 

Information 


None 


D 


Key 
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Model #8: 


Stem 


Stem: Mixed; Options: Constrained; Auxiliary Information: Table 


Mrs. Kary kept a record of participants in school activities and the total 
points some teams accumulated. 


Team 

Participants 

Total Points 

Red 

26 

762 

Green 

33 

978 

Yellow 

22 

641 

Blue 

29 

? 


Based on the information in the table, the Blue team's total number of 
points would most likely be 


A. 692 

B. 768 

C. 809 

D. 851 

Item Model Variables 

S| kept a record of participants in school activities and the total points 
some teams accumulated. 


Team 

Participants 

Total Points 

0-1 

0 

Round0*0 


0+7 

Round (0+ 7) 

0L3 

0-4 

Round 0-4)*0 

0L4 

0+3 

? 


Based on the information in the table, the[S^_4 team's total number of 
points would most likely be 


0 Range: "Blue", "Green", "Yellow", "Red", "Gray", "Brown", "Black", "White" 
S2 Range: "Mr. Kary", "Mr. Rogers", "Mr. Pitt" 

[IT] Value Range: 20 to 29 by 1 
[i^ Value Range: 29.00 to 29.99 by 0.3 
g Value Range: 29.00 to 29.99 by 0.3 
0 Value Range: 29.00 to 29.99 by 0.3 
0 Value Range: 29.00 to 29.99 by 0.3 


Elements 
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Options 


A. Round (([ii|-4) *[|3+7) 

B. Round (([n]-4) *0+29) 

C. Round (([n]* 12 + ([n]+ 7) * 13 + ([H]- 4) *0) / 3) 

D. Round(([n]+3)*[i5]) 


Auxiliary 

Information 


Table 


D 


Key 
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Model #9: 


Stem 


Elements 


Options 

Auxiliary 

Information 

Key 


X 23 ') 


Stem: Mixed; Options: Fixed; Auxiliary Information: None 

There are three radio stations in a town with about 60,000 potential 
listeners. Peter was hired to determine the approximate number 
of listeners each station had. He decided to survey 200 people. 

These 200 people are called a 

A. sample 

B. population 

C. frequency 

D. census 


Item Model Variables 

There are[IT| ^ in a town with a bout [i^ po tential S2.|^ was hired to 
determine the approximate number of^ each one had. He decided to 
survey[l3|people. 

These[i3]people are called a 


I Range: "radio stations", "TV stations", "sports teams" 

S2 Range: "listeners", "watchers" "fans" 

^ Range: "school", "campsite near Jasper", "community league" 

ImI Value range: 2 to 5 by 1 

B Value range: 20,000 to 80,000 by 1 0,000 

lisl Value range: 200 to 600 by 50 

As SI = "radio stations", then S2= "listeners" 

As SI = "TV stations", then S2= "watchers" 

As S3= "sports teams" then S2- "fans" 

A. sample 

B. population 

C. frequency 

D. census 

None 


A 


Jtl-a 
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Model #10: 


Stem 


Options 


Auxiliary 

Information 

Key 


Stem: Fixed; Options: Randomly Selected; 
Auxiliary Information: None 


There is a team crest on Henry's jacket that: 

• is a polygon 

• has more sides than a triangle but fewer than a hexagon 

• is not a quadrilateral 
What is the shape of the crest? 



Item Model Variables 

There is a team crest on Henry's jacket that: 

• is a polygon 

• has more sides than a triangle but fewer than a hexagon 

• is not a quadrilateral 
What is the shape of the crest? 



None 
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Generating Items from Models 

With 10 functional item models in mathematics, the second step was 
to develop a software engine that automatically creates and banks items 
for each model. This type of software serves as a proof-of-concept to dem- 
onstrate the practicality and feasibility of our item generation approach^. 
The software is called IGOR (Item GeneratOR). It was written in Sun 
Microsystems JAVA SE 6.0. The purpose of IGOR is to generate items for 
each model. A short description of the item generation software is pre- 
sented next. This description can also be found in the ‘Help' menu of the 
IGOR software. 


J-T-L-A 
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Getting Started 

This editor is used to generate item models that can be used to dynami- 
cally create item banks for use in test construction. An item stem can be 
defined, along with an optional set of variables, constraints, and response 
choices to generate a wide range of item models. 

The User Interface 

There are three components to the main screen of the editor window: 

The stem is the base from which each derived test item is created. Any 
text or formula entered here will appear as the stem in each test item. 

The elements are the variables and constraints used to generate each 
individual test item. Variables can consist of either a number range or a 
series of text values. 

The options are the possible answers to a given test item. They are 
classified as either key or distractor. 



The Elements and Options panels each contain three buttons. The first 
of these adds a new element or option to its panel. The second opens a 
window to edit the currently selected element or option. The third removes 
the currently selected element or option from the model. 


+ 
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Elements 

Elements in item models are used to create variety between different 
test items. This is done using variables, the values of which change from 
item to item, and constraints, which restrict the ways in which variables 
may be combined. 


Variables and Constraints 


There are two t)^es of variables in the item model editor. The first is 
the text variable. A text variable may have any number of possible lines of 
text, each of which must be matched with a corresponding numeric key. 
A key for a given value may be any desired integer, but no two values can 
have the same key. 




1 




Kei> 


value 

3 



i 

square 


5 







LiJL. 

C« I I Cancel ~] 


The second type of variable is the numeric variable. A numeric variable 
may take on any value between some minimum and maximum value, and 
increases by some fixed step size. 



Variables in an item model may have constraints placed on them, 
restricting the values they may take within a given test item. Constraints 
may be useful for eliminating some combinations of variables that yield 
nonsense items or items that are too similar to other items in a bank. 


Coretrant: !mmi > O M ; ^ 5) 


OK 


Caicd 
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Options 

Options represent the different possible choices available in any 
given test item. They may be classified as either keys, representing cor- 
rect responses, or distractors, representing incorrect responses, and may 
have either text or numbers as their values. Options may have restrictions 
placed on them so they change when a variable has different values. 


Tvp$: W 

O DstlKtOf 


Text: 


A rt>7/ Citinl^in Ay rtunnti^r nf tf?id ^r>d niimhf^f 

vanablfts. as wall as an/ number Df constraints upon 
UiDse variables. 



Type: O Key 

Cf: Distractor 


value; |[M 1-1 




OK 


Cancel 


Creating a Simple Item Model 

Creating an item model requires the following three steps: 

Define a Stem 

Define a stem for the item model. The model can contain variables, 
defined in the Elements panel, equations, and/or LaTeX formulas. 


Stem 

Ttie Sid* len-^ths of a pool are E01H. [P2I|. and ni3|l molers. Taspatthel/J/Viffi the flow 
rjit* |fi4]]i. p*r |pi]), iiaw mfiny hours will mis pool filled upi^ 
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Create Elements 

Create elements such as variables that change from item to item and 
constraints that restrict the values of these variables. A multiple- choice 
item model requires a stem, a key, and one or more distractors. 


6erwMt9 

Variibld inuirfric) 

Slapi 

TfiOh 3flfl 
S.D 

li^arlabld (nurr«rlc) 

[I ^ 



Slap 

5-0 

variable ^'ilurTifiriO 

3 


3(ltn^d 


2.D 

Variably ingrr^nc^ 

4 


3fio.Dto5aaa3LD 



Variabltd 

=1 

vaiu4 1 

Tiinils 

vaiui5 


Com lira nrf 

aGii^ 1 M = my ofi f [1511] ... 

IZlSLi] 


Create Options 

Create options for the item model. Options are divided into two groups. 
The first group, key, consists of the correct response for the test item. The 
second, distractors, consists of the incorrect responses for the test item. A 
multiple- choice item model requires at least one key and one distractor. 


DfVtttfIS 

(numericj 

i[iirui2ii‘PD*ioijci^i(!i4reoj 

Constraint 

= t 

(numsftq 

IPID *011211* PD *1 000 

cons-tnaint 

31 

□i£traclor 

IptmiJii'PDMDOCifinu 

con$trainc 

31 - 1 

Distrsclor (ni. marie) 

(ptB ‘HI2I]*PD^III4E 

Constraint 

31 -t 

DisiFacior (nimanc) 

fpipni^D ■ PE/ni4t + 10 

Constraint 

31 1 

Oistroclgr [nem^nt) 

|pilj'lIi?J]-pE'i0O0r|iiU ■ 5 

Constraint 

II 

Distraclor Cm marie) 

(pti]‘[ii:ai‘P[^iiiiD*5 

Constraint 

31 

Ui$iT^r:inr [nemont) 

ipi]j ■ Jl^ll’PL’ lOMP^Il*]] *1U 

Constraint 

B1 
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Options 

Stems, text variables, and text options may contain equations and for- 
mulas. They may be evaluated as numerical or logical values or used to 
generate an image of the equation in the test bank output. 

f —^<11 

J 1 -h 

f-U 


Equations are marked with /(to indicate the start of the formula and 
/) to indicate the end of the formula. 

LaTeX Formulas 

LaTeX formulas may be used to display mathematical content. Such 
formulas are marked with the characters / (to indicate the start of a for- 
mula and / to indicate the end of a formula). Supported LaTeX tags include, 
but are not limited to: 


\frac{numerator}{denonninator} 

1 

2 

\sq rt [root]{a rg u merit} 


\cdots, \ddots, \ldots 

* * * * 

Greek letters \alpha to \omega 

afi . . . lj 

Meftarrow, \Leftarrow, \uparrow, Meftrightarrow ... 


\cap, \cup 

n u 


Our program uses JMathTex for LaTeX formulas and supports all 
LaTeX features in the JMathTex library. 

Once an item model is created, a test bank can be generated and saved 
to an HTML file. This file is produced using the Generate menu in the 
editor. 



Fric Edit Ocncriilt Mdp 


Stem KoinJiW me 

rr«tn cur rcfii iimcfei 
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To generate items from a model, the user will be presented with the 
dialogue box shown below. In this box, the user must specify the item 
model file, the item bank output file, and the answer key file. If the option 
‘Create answer key' is not selected, then the resulting test bank will always 
display the correct answer as the last option (or alternative). If the option 
‘Create answer key' is selected, then the resulting test bank will randomly 
order the options. Once the files have been specified in the dialogue box, 
the program can be executed by selecting the ‘Generate' button. 



When IGOR was applied to the 10 mathematics item models, 331,371 
items were created. Generation capacity is dependent on several factors 
including the model, the number of elements in the stem of the model, 
and the range specified for the elements. For this demonstration, item 
generation ranged from a low of 8 items for model 6 to a high of 202,860 
items for model 5. A summary of each model and their generation capacity 
is presented in Table 2. 

Table 2: Mathematics Items Generated from 1 0 Item Model Types 


Mathematics Item Model 

Number of Items Generated 

1 (Independent; Randomly Selected) 

1,280 

2 (Independent; Constrained) 

27 

3 (Independent; Fixed) 

16 

4 (Dependent; Randomly Selected) 

16 

5 (Dependent; Constrained) 

202,860 

6 (Dependent; Fixed) 

8 

7 (Mixed; Randomly Selected) 

364 

8 (Mixed; Constrained) 

122,880 

9 (Mixed; Fixed) 

3,780 

1 0 (Fixed; Randomly Selected) 

140 

Total 

331,371 


J-T-L-A - 



Developing a Taxonomy of Item Model Types to Promote Assessment Engineering 


Gierl, Zhou, Alves 


{ 32 


Summary 

An item model (LaDuca et al., 1986; Bejar, 1996, 2002; Bejar et al., 
2003) serves as an explicit representation of the variables in an assess- 
ment task. For a multiple- choice item, the developer can manipulate infor- 
mation in the stem, options, and auxiliary information. In the current study, 
we introduced an item model taxonomy and applied the taxonomy to dif- 
ferent content areas. We also developed and applied a software program 
called IGOR to the math item models to generate instances for each model. 
Singley and Bennett (2002, p. 366) claimed that an item model serves as 
“a schematized description of a class of questions from which draft test items 
are automatically generated for the test developer to review and revise!' In 
other words, item modeling can supplement, rather than replace, tradi- 
tional test development procedures by offering a systematic and strategic 
approach for manipulating content so items can be produced automati- 
cally. This type of integrated approach to test construction, where content 
expertise and technology are efficiently combined to direct the design, 
development, and production of test items, is one example of how assess- 
ment engineering can alter educational and psychological testing. The role 
of the content specialist is critical for the creative task of designing and 
developing meaningful models. At the same time, the role of technology 
is critical for the algorithmic task of combining large number of elements 
in each model to produce items which, in turn, are stored in banks. By 
combining content expertise and technology, item modeling could be used 
to generate content for an entire test. Item modeling can also be used to 
supplement existing test content by promoting generative processes for 
some item types which, when combined with items created using the more 
traditional approach, could produce the content for the final test form. A 
testing program may design two item models from our taxonomy to gen- 
erate some operational items. But, through development and experience, 
item modeling and the supporting generative processes may expand to 
account for larger percentages of test content over time. 

Theory and Item Modeling 

The approach to item modeling, as presented in this paper, is very 
practical. We omitted the cognitive model step, identified first in the 
assessment engineering framework, because our focus was on item model 
design. Our sample models were developed from weak theory by reviewing 
large numbers of items from previous test administrations to illustrate 
how the four stem and three option categories could be combined to pro- 
duce diverse yet functional models in a practical testing context. The prac- 
ticality of weak theory can also be used to generate items automatically, 
as we noted earlier, because design guidelines can produce parent models 
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that, when combined with specialized statistical procedures, yield item 
parameter estimates for the isomorphic instances. 

Despite the practicality of weak theory, strong theory is still the 
preferred approach. With strong theory, calibrated items are generated 
automatically using the design principles articulated in a cognitive model 
(Drasgow et al., 2006). A cognitive model in educational measurement 
refers to a “simplified description of human problem solving on standard- 
ized educational tasks, which helps to characterize the knowledge and 
skills students at different levels of learning have acquired and to facili- 
tate the explanation and prediction of students' performance” (Leighton 
& Gierl, 2007, p. 6). The cognitive model provides a detailed description 
of the variables that affect examinee performance which, in turn, can help 
pinpoint the item difficulty elements. Although few comprehensive and 
practical cognitive models exist for test development, educational and 
psychological researchers are beginning to identify and articulate these 
models as well as to create psychometric methods for evaluating their 
statistical properties. For instance, Leighton, Gierl, and Hunka (2004) 
introduced a procedure called the attribute hierarchy method (AHM), which 
is used to classify examinees' test item responses into a set of structured 
attribute patterns associated with different components from a cognitive 
model of task performance. Attributes include different procedures, skills, 
and/or processes that an examinee must possess to solve a test item. The 
attributes are structured using a hierarchy so the ordering of the cognitive 
skills is specified. As a result, the attribute hierarchy serves as an explicit 
cognitive model. This model, in turn, provides a framework for designing 
item models and for linking examinees' test performance to specific infer- 
ences about psychological skill acquisition. 

The AHM has been used to develop and analyze a cognitive diagnostic 
assessment in high school algebra (e.g., Gierl, Wang, & Zhou, 2008). 
Cognitive diagnostic assessment is a form of testing that employs a cog- 
nitive model to first develop items that measure specific knowledge and 
skills and then uses this model to direct the psychometric analyses of the 
examinees' item response patterns to promote specific diagnostic infer- 
ences. The AHM can also be used for computer adaptive testing (Gierl & 
Zhou, 2008). Computer adaptive testing is an innovative form of assess- 
ment that matches the difficulty of a test item to the ability estimate 
for an examinee. The matching is accomplished by first presenting an 
examinee with an item of average difficulty and then, depending on the 
examinee's response, an item of greater or lesser difficulty is presented 
until the algorithms controlling item administration meet a specified 
stopping criterion. 
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Both cognitive diagnostic assessment and computer adaptive testing 
make heavy demands on the items in the bank. One benefit of using a cog- 
nitive model-based procedure, like the AHM, lies in its facility to guide item 
development. The AHM requires the developer to specify a reduced inci- 
dence matrix of order (k, i) where k is the number of attributes and i is the 
number of items specified in the hierarchy. The reduced incidence matrix 
serves as the cognitive test specifications because it identifies all possible 
attribute-by-item combinations in the cognitive model. Item models can 
therefore be developed to measure each attribute-by-item combination 
(see Gierl, Wang, et al., 2008, p. 34). For example, three items are required 
to measure three algebra attributes, Al to A3, in Figure 3 (next page) 
(the cognitive model in Figure 3 contains 9 attributes but, for illustra- 
tion purposes, only the first three attributes are presented). Item 1 
measures attribute Al, which includes basic arithmetic operation skills; 
item 2 measures attribute A2, which includes knowledge about the prop- 
erties of factors in addition to basic arithmetic operation skills (i.e., Al); 
item 3 measures attribute A3, which includes the application of factoring 
in addition to the properties of factors (i.e., A2) and basic arithmetic opera- 
tion skills (i.e., Al) (this algebra model is described in more detail in Gierl, 
Wang, et al., 2008). 
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Figure 3: Stages 1 and 2 in the Assessment Engineering Framework 
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In addition to creating each attribute-specific item, multiple instances 
of these items must also be created to produce a large functional bank for 
continuous diagnostic and adaptive testing. Research on item modeling, 
as described in the current paper, can support this process where a parent 
item model is developed for each attribute in the cognitive model and 
isomorphic instances are generated. Hence, the parent model yields large 
numbers of items for each attribute in the cognitive model that will func- 
tion similarly within or between test forms. In the Figure 3 example, the 
cognitive model prescribes the hierarchy of skills in step #1. The cognitive 
model is also used to specify the attribute structure for item development 
in step #2. Taken together, steps #1 and #2 constitute stage 1 in an assess- 
ment engineering framework. In step #3, a parent item model is created 
to measure each attribute according to the specific hierarchical ordering of 
increasing cognitive complexity. The appropriate elements in the stem and 
options are then manipulated in step #4 to generate isomorphic instances 
for each item model. These last two steps, together, serve as stage 2 in 
assessment engineering. In sum, both the cognitive and item modeling 
stages are required to produce an efficient item generation system necessary 
for the development and production of large numbers of items - these 
items, in turn, provide the foundation for a modern 21^^ century assess- 
ment program where continuous diagnostic and adaptive testing are not 
only possible, but potentially viable. 
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Endnotes 

1. Item models have been characterized in different ways. For example, they have been 
described as schemas (Singley & Bennett, 2002), blueprints (Embretson, 2002), 
templates (Mislevy & Riconscente, 2006), forms (Hively, Patterson, & Page, 1968), 
and shells (Haladyna & Shindoll, 1989). 

2. For Glas and van der Linden (2003) isomorphic instances are called item clones. 

3. It is common to pair conceptual and technological advances in the item generation 
literature. For example, Singley and Bennett (2002) developed an item generation 
program called Mathematics Test Creation Assistant to illustrate how schema theory 
could be used to automatically produce math items. 
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Auxiliary 
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Key 



Appendix 

The appendix contains a second set of item model examples, as out- 
lined in Table 1 (page 12), drawn from diverse content areas. 

(Biology) Stem: Independent; Options: Randomly Selected; 

Auxiliary Information: None 


After a head injury, a nnountain biker was assessed by a doctor. One of the 
tests the doctor did was to gently tap just below her kneecap. Also, he 
shone a light into each of her eyes and made observations. In both tests, 
he was trying to quickly rule out any neurological damage. 

Assume that the biker's nervous system was not injured and that the 
biker's body responded in a normal way to the gentle tap. The neurological 
pathway that was followed when this response was elicited was 

A. receptor, sensory neuron, interneuron, motor neuron, effector 

B. effector, sensory neuron, interneuron, motor neuron, receptor 

C. receptor, motor neuron, interneuron, sensory neuron, effector 

D. effector, motor neuron, interneuron, sensory neuron, receptor 


Item Model Variables 

After a head injury, a SI was assessed by a doctor. One of the tests the 
doctor did was to gently tap just below her kneecap. Also, he shone a light 
into each of her eyes and made observations. In both tests, he was trying 
to quickly rule out any neurological damage. 

Assume that the|^'s nervous system was not injured and that the 
SI 's body responded in a normal way to the gentle tap. The neurological 
pathway that was followed when this response was elicited was 


[si] Range: "mountain biker", "truck driver", "driver", "motorcycle operator" 


Key: receptor, sensory neuron, interneuron, motor neuron, effector 
Distractors : all other combinations of the neurological processes 


None 


A 
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(Social Studies) Stem: Independent; Options: Constrained; 
Auxiliary Information: Map 



Item Model Variables 



[IT]Value Range: 22-24 by 1 
[j^Value Range: 37-39 by 1 

13 Range: "Athens", "Delphi", "Corinth", "Olympia", "Sparta" 

As SI = "Athens", then 11=24, 12=38 

As SI = "Delphi", then 11=23, 12=39 

As SI = "Corinth", then 11=23, 12=38 

As SI = "Olympia", then 11=22, 12=38 

As SI = "Sparta", then 11=23, 12=37 

A. [n]°N and[]|°E 

B. ([n]+2)°N and([g+4)°E 

C. [i^°N and[n]°E 

D. ([ig+4)°N and ([n]+2)°E 


Map of Ancient Greece 


C 
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(Language Arts) Stem: Independent; Options: Fixed; 
Auxiliary Information: Reading Passage 


The word "it" as used in line 1 9 refers to 


A VIO»QtJ|T0 IPi THPrABI\ 




Ahhi-iuf |i tvJi hcf. 

A. 

water 


hc¥. iiTuuih liilrf , 

111 bed vktodovi. 




and 

B. 

blood 

J 

dw will CORK 
muming: .md ziufin^ 

C. 

food 


pver y-Dfr b«l, 
Ytita dap IIH: axf 

D. 

air 

W 

due’* in ywr btir 
v(\iM 


Ychi qm»cV ycHor btjd. 




hu1 dK '1 dexJi — 
siw'^wnht rafKr. 



IS 

Sbe's cni for Mood — 
yiMix tii^ IpcikL 
and dif Thill f/ii it. in [Jk cf\± 
'bfiilil iE friM bobciLiijj pyuiT, 




ilinklct^ ]i 




With a fiv, fi, fn ind conln^t'J fiim 




sht Mpi II 
wlufc hoy drcaort. 






Item Model Variables 

The word "it" as used in ^ refers to 


^ Range: "line 17", "line 18", "line 19", "line 21" 


Key: blood 

Distractors : water, food, air 


The text "A mosquito in the cabin". Author: Myra Stilborn. 


B 
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(Language Arts) Stem: Dependent; Options: Randomly Selected; 
Auxiliary Information: Reading Passage 


The word "converted" in the last 
sentence of Step 4 nneans 

A. added 

B. exposed 

C. changed 

D. compared 


A. r 4 -HUHMTU 



I riv ilwH' ntewi. thcTi fttavk -a rkr irdar Viui 

p.tf k» M> -NK' H .k Liwd- u> uUliV kOI 

(M ■.* ^ m i^l Um y fiUkdlMj u .kJ 

ki >WiiM ^ h.Jik 5 i:r<' 


k Kmp Ikf^ J Jfail4 Har * lf» i— l».<tkKlLM# Uf 

likH- ^lukii fkld -CM piulil^vlL Aji ih «mJ«l tatr nwidik. 

mwm. hi pMfc^ fk-rf Tr^rtJjH id i.d 

-knr rt l Tk l l ^T lik- 




Item Model Variables 

The word"[s^ 

"in the last sentence of^ 

means 


^ Range: "converted", "perk up" "fertilize" 

H Range:"Step4"or"Step5" 

As SI ^"converted" then S2="Step 4" 

As Sl="perk up", or "fertilize", then S2="Step 5" 


As 5/ = "converted" 

As SI = "perk up" or "fertilize" 

Key = "changed", "altered", or 

Key = "lift up", "boost", "uplift", "enrich". 

"transformed" 

or "raise" 

Distractors="added", "exposed". " 

Distractors= "prepare", "produce", "fix". 

CO m pa red ", "co m bi n ed ", "i n se rted ", 

"build up", "water", "adjoined", 

"adjoined" "showed", "contrasted" 
or "assessed" 

"showed", or "contrasted" 


The text "Make a composter". Author: Beth Savan. 


One of the key alternatives presented in the option section will be 
randomly selected. 
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(Chemistry) Stem: Dependent; Options: Constrained; 
Auxiliary Information: Graph 


The researchers made a graph of data 
collected along a river downstream from * ' 

the plant. They know that rainbow trout « ■ 

cannot live in water with a pH of 5.5 or less. ^ , . 

pH oi Water Alon^ 1 hv R i ver 

Based on information in the graph, rainbow j • 

trout in the river could survive downstream ^ 


from the plant at a distance of < 

1 S 10 I? 20 25 

A. 0 to 10 km 

Dctavcc- rrmn fxlarf [Km) 

B. 5 to 15 km 


C. 1 0 to 20 km 


D. 1 5 to 25 km 


Item Model Variables 


The researchers made a graph of data collected along a river 
downstream from the plant. They knowthatj^ cannot live in 
water with a pH of[n]or less. 


Based on information in the graph, in the river could survive 
downstream from the plant at a distance of 


[iT] Value Range: 5.5, 5.6, 5.4, or 6.7 

[?i] Range: "rainbow trout", "fathead minnow", "pearl dace", "lake trout" 
As Sl="rainbow trout", then 11=5.5 
As Sl="fathead minnow", then 11=5.6 
As Sl="pearl dace", then 11=5.4 
As Sl="lake trout", then 11=6.7 
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(Biology) Stem: Dependent; Options: Fixed; 
Auxiliary Information: Picture 


Within a lake, you observe two different species offish. 



The process nnost likely responsible for the development of different 
species offish is 

A. Artificial selection 

B. Artificial breeding 

C. Selective breeding 

D. Natural selection 

Item Model Variables 

Within a|^, you observe two different species of S2 . 

[insert Picture[^] 

The process most likely responsible for the development of different 
species of S2 is 

m Range: "jungle", "lake", "mountain", "sea" 

S2 Range: "monkey", "fish" "pine", "turtle" 

[rj Range: picture of two species of monkeys, picture of two species offish, 
picture of two species of pines, or picture of two species of turtles 
As Sl="jungle", then S2="monkey", P1=picture of two species of monkeys 
As Sl="lake", then S2="fish", P1=picture of two species offish 
As Sl="mountain", then S2="pine", Pl^picture of two species of pines 
As SI ="sea", then S2="turtle", PI =picture of two species of turtles 


A. Artificial selection 

B. Artificial breeding 

C. Selective breeding 

D. Natural selection 


Picture of two species of monkeys; Picture of two species offish; Picture of 
two species of pines; Picture of two species of turtles 



D 
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(Physics) Stem: Mixed; Options: Randomly Selected; 

Auxiliary Information: None 

While driving to the mountains, you see a glider. As the glider flies across 
a meadow, you see it rise sharply. This lift is created by the method of heat 
transfer known as 

A. transmission 

B. conduction 

C. convection 

D. radiation 


Item Model Variables 

While driving to the mountains, you see a |^. As the[^ flies across a 
you see it rise sharply. This lift is created by the method of heat transfer 
known as 


0 Range: "glider", "hang glider", "bunch of leaves", "kite", "hawk"" 
^ Range: "meadow", "pasture" 

As Sl="glider"or"hang glider", then S2="meadow"; 

As SI ="bunch of leaves", "kite", or"hawk" then S2="pasture" 


Key: radiation 

Distractors : transmission, conduction, convection, diffusion, absorption 


None 


D 
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(Mathematics) Stem: Mixed; Options: Constrained; 
Auxiliary Information: None 


Ann has paid $ 1 525 for planting her lawn. The cost of lawn is $45/m^. 

Given the shape of her lawn is square, what is the side length of Ann's lawn? 

A. 5.8 

B. 6.8 

C. 4.8 

D. 7.3 


Item Model Variables 

Ann has paid $[n]for pla 
the shape of her lawn is 

mti 

SI 

ng her lawn. The cost of lawn is $ 12/m^. Given 
, what is the[^ of Ann's lawn? 


E] Value Range: 1525 -1 675 by 75 
[1T| Value Range: 30 or 40 
[Til Range: "square" or"circular" 

[S^ Range: "side length" or "radius" 

As SI = "square", then S2 = "side length" 
As SI = "circular", then S2 = "radius" 


As 51 = "square" 

As 51 = "circular" 

A. = a/^T/2 

A. = V'/,2*3.14 

B. = a/^T/2 + 1 

B.= y/l//2*3.14 +1 

C.= a/^V/2 

C.= VV;2*3-I4 - I 

D.= VTT + 1.5 

D.= VV;2*3.14 +1.5 
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Model #9: 


Stem 
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Auxiliary 

Information 
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(Biology) Stem: Mixed; Options: Fixed; 
Auxiliary Information: Graph 


On a newly formed island, successful 
populations of grasses and a species 
of mouse appeared. Later, a species of 
hawks flew in. The hawks feed on mice. 

The population levels of mice and hawks 
are represented in the graph. 

In 1 991 , the data for the mice indicates that 

A. r is negative because b<d 

B. r is negative because b> d 

C. r is positive because b<d 

D. r is positive because b> d 



r = per capita population 
growth rate (b-d) 
b= per capita births 
d- per capita deaths 


Item Model Variables 

On[|l] successful populations of grasses and a species of^ appeared. 
Later, a species of ^ flew in. The S3 feed on |^.The population levels of 
^and S3 are represented in the graph. 

In0, the data for the ^indicates that 
0] Range: "1990", "1991", "1994" 

SI Range: "a newly formed island", "distant forest", "isolated jungle" 

^ Range: "worms", "beetles" "mice", "snakes" "fish" "lizards", "insects", "bugs", 
"frogs" 

S3 Range: "hawks", "eagles", "ravens" 

As S3="hawks" then S2="worms" "beetles", "mice" "snakes", or "frogs" 

As S3="eagles", then S2="fish", "snakes", or"lizards" 

As S3="ravens" then S2="lizards" "insects" "bugs", or'Trogs" 


Graph with Yearly Populations 


A 
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Model #10: 


Stem 


Options 


Auxiliary 

Information 

Key 


(Architecture) Stem: Fixed; Options: Randomly Selected; 
Auxiliary Information: None 

What is one purpose of a bid repository? 

A. To distribute bid documents and addenda 

B. To administer construction contracts 

C. To be a storage facility for typical construction details 

D. To ensure substantial performance 


Item Model Variables 

What is one purpose of a bid repository? 


Kev : 

"To distribute bid documents and addenda", 

"To manage the distribution of bid information to trade contractors", or 
"The timely collection of trade contractor bids" 

Distractors : 

"To administer construction contracts" 

"To be a storage facility for typical construction details", 

"To ensure substantial performance", 

"To extend the tendering process timeline", 

"To assist Owners in the selection of architects" 

"To collect bonds from prospective contractors", 

"To tabulate and monitor LEED credits" 

"To assist the Owner in identifying a financial partner", or 
"The fund construction deficiencies" 


None 


A 
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